Modification of Zipf-Mandelbrot Law for Text Analysis using Linear Regression
نویسندگان
چکیده
منابع مشابه
Citations and the Zipf-Mandelbrot Law
p1, p2, and p3 all being constants. The same inverse power-law statistical distributions were found in embarrassingly different situations (e.g., [6, 7]). In economics, it was discovered by Pareto [8] over 100 years ago and states that incomes of individuals or firms are inversely proportional to their rank. In less formal words [9], “most success seems to migrate to those people or companies w...
متن کاملMajorization, Csiszár divergence and Zipf-Mandelbrot law
In this paper we show how the Shannon entropy is connected to the theory of majorization. They are both linked to the measure of disorder in a system. However, the theory of majorization usually gives stronger criteria than the entropic inequalities. We give some generalized results for majorization inequality using Csiszár f-divergence. This divergence, applied to some special convex functions...
متن کاملOn the Law of Zipf-Mandelbrot for Multi-Wort Phrases
The paper studies the probabilities of the occurrence of m word phrases (m=2,3, ...) in relation with the probabilities of occurrence of the single words. It is well-known that, in the latter case, the law of Zipf is valid (i.e. a power law). We prove that in the case of m word phrases (m22) this is not the case. We present two independent proofs of this. We furthermore show that in case we wan...
متن کاملMinimum cost and the emergence of the Zipf-Mandelbrot law
This paper illustrates how the Zipf-Mandelbrot law can emerge in language as a result of minimising the cost of categorising sensory images. The categorisation is based on the discrimination game in which sensory stimuli are categorised at different hierarchical layers of increasing density. The discrimination game is embedded in a variant of the language game model, called the selfish game, wh...
متن کاملBeyond the Zipf-Mandelbrot law in quantitative linguistics
In this paper the Zipf-Mandelbrot law is revisited in the context of linguistics. Despite its widespread popularity the Zipf–Mandelbrot law can only describe the statistical behaviour of a rather restricted fraction of the total number of words contained in some given corpus. In particular, we focus our attention on the important deviations that become statistically relevant as larger corpora a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Indian Journal of Science and Technology
سال: 2017
ISSN: 0974-5645,0974-6846
DOI: 10.17485/ijst/2017/v10i3/110616